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Abstract 

We generalize a construction of Dunkl, obtaining a wide class intertwining functions on 
the symmetric group and a related family of multidimensional Halm polynomials. Following 
a suggestion of Vilenkin and Klymik, we develop a tree- method approach for those intertwin- 
ing functions. We also give a group theoretic proof of the relation between Hahn polyno- 
mials and Clebesh-Gordan coefficients, given analytically by Koornwinder and by Nikiforov, 
Smorodinskii and Suslov. Such relation is also extended to the multidimensional case. 

1 Introduction 

One of the most fruitful method to study special functions is to use representation theory and 
harmonic analysis. In particular, discrete orthogonal polynomials can be treated by mean of 
the representation theory of finite group, and a lot of work in this direction was made by 
Delsarte, Dunkl and Stanton; see [24] for a useful survey. In [7] Dunkl developed the frame- 
work of intertwining functions and in [6j he used it to study Hahn polynomials. Indeed, the 
Sn-m X Sm — Sn-h X S'/i-intertwining functions on the symmeric group are naturally expressed 
in terms of these polynomials. In [9] he studied the Sa x Sb x Sc — Sn-m x Sm intertwining 
functions on the symmetric group Sn (where n = a + b+ c), expressing them in terms of a class 
of two dimensional Hahn polynomials. Those polynomials had been introduced by Karlin and 
McGregor in [15] . 

The aim of the present paper is to extend Dunkl's results and to study the Sa^ x Sa2 x 
• • • X Sa^ — Sn-m X ^m-intertwining functions on Sn (ai -|- 02 + • • • + a/i = n). Following a 
suggestion of Vilenkin and Klymik (see pTT], p. 505), we develop the tree method for those 
intertwining functions and for the related Hahn polynomials. The key ingredient is an explicit 
Littlewood- Richardson rule for the irreducible representations of the symmetric group associated 
to two rows Young tableaux. The resulting Hahn polynomials are more general than those in 
|15j . that correspond to a special choice of the tree. It is well known that Hahn polynomials 
are related to the Clebsch-Gordan coefficients for SU{2); see [161 US US 123]. We give a group 
theoretic approach to this relation and extend it to the multidimensional case. Indeed, the 
relation between Clebsch-Gordan coefficients for SU{2) and the representation theory of the 
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symmetric group was investigated in [14:\ [26} [27j , but in those references no connection is found 
with Dunkl's intertwining functions and Hahn polynomials (and with the analytical formulas in 
[T6l [TBI [T9l [23]). In view of these results, we do not compute connection coefficients between 
different bases of intertwining functions (this is the main problem in every tree method): they 
can be obtained by the equivalent theory on SU{2). 

The plan of the paper is the following. In section 2, we review the Hahn polynomials (using 
Dunkl's renormalized notation) and the Gel'fand pair (Sn, Sn-m x Sm) (also called the Johnson 
scheme). Then we give an explicit Littelwod. Richardson rule for the spherical representation of 
the Johnson scheme. This rule is expressed in terms of Hahn polynomials and Radon transforms 
and our formulas generalize those for the intertwining functions in [6]. In section 3, we develop 
the tree method for the multidimensional Hahn polynomials and obtain a basis for the Sa^ x 
Sa2 X ■ ■ ■ X Sa^ — Sn-m X 5m-intertwining functions on the symmetric group. We also give a basis 
for the Sn-m x Sm — Sai X X • • • X -intertwining functions, that are more directly related to 
the Clebsch-Gordan coefficients. Though these functions may be obtained by the simple change 
of variable g —>■ g~^, their study requires the use of the theory of induced representations. 
As a by-product, we give a group theoretic proof of the Regge's symmetries for the Hahn 
polynomials. In section four we introduce Clebsch-Gordan coefficients and show their relation 
with the intertwining functions on the symmetric group. We make use of the Schur-Weyl duality 
in the form developed by James in p^, which is quite suitable for our purposes. First we analyzed 
the case of the tensor product of two irreducible 5C/(2)-representations, connecting it with the 
theory of Sn-m x Sm — Sn-h X S'ft-intertwining functions on 5„. Then, in the last subsection, we 
show that the tree method for the intertwining functions on the symmetric group is equivalent 
to the tree method for the Wigner 3nj-coefficients for SU{2). 

Another generalization of the results in [9] is in [22], where an orthogonal basis for the 
Sa X Si) X 5c-invariant functions in the irreducible representation 3°^'^''^ of S^ (where N = 
a + h + c = a + (5 + ^) is obtained. 

2 Hahn polynomials and the Johnson scheme 
2.1 Hahn polynomials 

We recall the basic properties of the one dimensional Hahn polynomials, using Dunkl's renor- 
malized notation; [6l[9l[22]. For m,a,b,c,x integers satisfying: 

< c < a + b, < m < min{a, b, c, a + b — c} and max{c — 6, 0} < x < min{a, c}, (1) 
they are given by the formula: 






m 
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) 



{h - m + l)j{-x)j{a - m + l)m-j{x — c). 



m-j , 



J =max{0, x—c+m} 



where {x)k = x{x + l){x + 2) ■ ■ ■ {x + k — 1). Now we list some of their properties. 
Transformation formula: 
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min{c,j/} 



x=ma,x{0,c—d+y} 

Orthogonality relations: 



^-^ \x J \c — X J \c — mj 



max{a,c} 

E 

a;=min{0,c— 6} 



• \ ^ J \ ''^ J 

a + h — m+1. ^, ^. , -,N / , -,N 

X ; -(a - m + l)m(o - m + l)m(c - m + l)m(a + 5- c- m + l) 

a + — 2m + 1 



(3) 

Particular values: for c = m we have 

Ern{a, b, m, x) = (-l)"'-^m!(a - m + l)m-x{b - m + 1)^. (4) 

For the difference relations, we refer to [6]; we will need the following particular case of (3.10) 
of [6]: for m = c + 1 we have: 

(a — x)Em{a, b, m, X + 1) + {x + b — m + l)Em{a, b, m, x) = 0, (5) 
as can be checked directly from 

2.2 The Johnson scheme. 

Fix a positive integer n and denote by Sn the group of all permutations of the set {1,2,..., n} 
(the symmetric group). For 1 < m < n, denote by the space of all m-subsets of {1, 2, . . . , n}; 
Sn acts on and Qm — Sn/{Sn-m X Sm)-, as homogeneous space. Following the standard 
notation in the representation theory of denote by M""™'™ the permutation module of all 
complex valued functions defined on O^- The space M""™'™ is endowed with the natural scalar 
product (/i,/2) = E^gn. fii^)M^, for /i,/2 G M""™'™. For A G ^m, Sa will indicate the 
Dirac function centered at A, that is Sa_{B) = 1 if ^ = i3, 5a{B) = if A ^ B. Now we 
introduce the Radon transforms d,d*: ii A £ Qm, we set 

d6A = ^ Sa\{x} and d*6A = ^ Sau{x}- 

xeA x^A 

We also introduce a particular notation for the powers of d*: 

Rq = j — , that is Rq^A = ^B- 

B£flm + q '■ 
ACB 

The operator d intertwines the permutation modules jVf""™'"* and and d* is 

the adjoint of d. Clearly, d, d* ,Rq may be defined on each for any the value of n and 

m; to simplify notation, we will not indicate the space on which they are acting. In the following 
lemma, we collect some basic properties of the operators d,d*,Rq [31 [H [5l [6| [22]. 
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Lemma 2.1. /// e M""™'™ then 

1- ifl<q<n — m then dRqf = Rqdf + {n — 2m — q + l)Rq-if; 

2- ifl<p<q<n — m and df = then {dy Rgf = (n — 2m — q + l)pRq-pf. 

In the following Theorem, we give the decomposition of M^~^'^ into irreducible Sn-representations 

laHElE]. 

Theorem 2.2. For < A; < n/2, set 5'"-'=''= = M"-'=''= n Kerd. 

1. S^~'^'^ is an irreducible Sn representation and its dimension is equal to (^) — (^";^)- 

2. IfO<m<n, 0<k< min{n — m, m} and fi, f2 G S'""'^''^, then 

{Rm-kfl, Rm-kf2)M"-"^'"^ = ( , ) (/l ) /2)m"-'='* 



m — 

and therefore Rm-k is injective from S^~^'^ to M"-~"^'"^ ; 

3. 

min{n— m,m} 

is the decomposition of M^~"^''^ into irreducible Sn-representations; 
2.3 An explicit Littlewood-Richardson rule for 5'"-'= '= 

In this section, we study the restriction Res^" ^^^^ S""" ' . We introduce the following notation: 
if X C {1, 2, ... , n}, \X\ = t and < j < t, then M*-^'^(X) and S^-^'^{X) denote the spaces de- 
scribed in the preceding section constructed by mean of the j-subsets of X. Set ^4 = {1, 2, . . . , h}, 
B = = {h + I, h + 2 . . . ,n} and suppose that Sn-h x is the stabilizer of A. 

First note that the map n^iA) x QtiB) 3 {X,Y) X UY identifies ^siA) x QtiB) with 

min{k,n—h} 

{Z G Qs+t ■■ \Z n A\ = s,\Z n B\ = t}. Moreover, = U [^lk~iiA) x is the 

i=max{0,fc— h} 

decomposition of Qk into Sn-h x 5'/i-orbits (in the present paper, ]J denotes a disjoint union). 
Form this we immediately get a preliminary decomposition: 

min{fc,n— ?i} 

Res|;;_,^x5^M"-'='^ = M'^-'=+'''^-'(.4)0M"-'^~'''(S). (6) 

i=max{0,fc— /i} 

Note that, in our notation, Sx ^ = ^xuY- Moreover, it is easy to see that ii (f> tp G 
^n-k+i,k-i^j^^ Nr'^'^\B), then 



'0 V] = (#) «'V' + <^® (#); (7) 
in the notation of [6], this is just the usual differentiation rule. Moreover, [3] in Theorem 12.21 
ensures us that M^~^~^^'^~\A)®M^~^~^'\B) contains a subspace isomorphic to S^~'^''^(E)S'"'~'^~^'^ 
if and only if max{i — h + k, j} < I < mm{k — i,n — h — j}. In the following Lemma and the 
subsequent Corollary, we give a characterization of the subspace of Res ^" ^xSh^'^~ isomorphic 
to 5'^-*'* ® S^'-^-^'K 
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Lemma 2.3. Suppose that < i < h/2, < j < {n - h) /2 and that 4> G 5'''"*'*(A), t/; G 
S^~^^^'^{B). Then the equation (in the unknown coefficients ai's) 

min{k—i,n—h—j} 

d Yl al{Rk-l-^(|)) ^ {Ri-jij) = (8) 

l=max{i—h+k,j} 

has a nontrivial solution if and only ifk — h<j — i<n — h — k and i + j < k. If these 
conditions are satisfied, then all the solutions are obtained by setting ai = aEk-i-j{n — h — 
2j,h — 2i,k — i — j,l — j), I = j,i + 1, . . . ,k — i, a arbitrary constant. 

Proof. It is just an application of ^ and Lemma l2.ll We can divide the study of ([8]) in four 
cases, according with the values of the limits in the sum. 

In the first case we take i — h + k < j and k — i < n — h — j, that is k — h<j — i<n — h — k; 
clearly, we must also have i + j < k. Now 

k—i 

d'Yai{Rk-i-i(p) <X) (Ri-jip) 

k—i—l k—i 

= al{h-k-i+l + l){Rk-l-^-l(t>)(^{Rl-j^)+ ^ ai{n-h-j-l + l){Rk-i-i<p)®{Ri-j-iil^) 

1=3 l=j+l 

k-i-l 

= ^ [a,(/i - - z + / + 1) + ai+i{n -h-j- l)]{Rk-i-i-i<P) ^ {Ri-j^)- 

1=3 

(9) 

Therefore ^ is satisfied if and only if 

ai{h — k — i + l + l) + a/+i(n — h — j — I) = 0, I = j, j + 1, . . . , k — i — 1, 
and this is solved by setting ai = Ek-i-j{n — h — 2j, h — 2i, k — i — j,l — j) (see ([5])). 

In the second case, we take j < i — h + k and k — i<n — h — j. Now we have 

k—i 

d Y ai{Rk-i-i4> ® Ri-ji)) 

l=i—h+k 

k—i—l k—i 

= Y ai{h-k-i+l+l){Rk-i-i-i<PMRi-j'il^)+ Y ai{n-h-j-l+l){Rk-i-i4>MRi-j-itp) 

l=i—h+k i—h+k 
k-i-l 

= Y [ai{h - k - i + I + I) + ai+i{n - h - j - l)]{Rk-l-^-l(t)) ® {Ri-ji)) 

i—h+k 

+ ai-h+k{n + k-i- j + l)iRh(p) (Ri-h+k-j-iip) (10) 
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and this is again equivalent to ai{h — k — i + I + 1) + a/+i(n — h — j — I) = 0, but now we 
have the extra condition aj_/j+fc = 0, and therefore we have only the trivial solution. 

In the same way, it easy to show that also if we take j > i — h + k and n — h — j < k — i, or 
j < i — h + k and n — h — j < k — i, we have only the trivial solution. 

□ 

An immediate consequence is an explicit form of Littlewood-Richardson rule for S'^^^'^ (see 
|12| for the general rule). 

Corollary 2.4. The multiplicity of S^-'''(^S'^-^-^'^ in Res^" ^n-fc.fe ig ^.q^al to 1 ifk-h < 
j — i<n — h — k and i + j < k; otherwise it is equal to 0. 
In particular, 

where the sum is over all i,j satisfying the above conditions (and the decomposition is mul- 
tiplicity free). Moreover, the map 



k—i 

(/)(g)V I — ^ '^Ek-i-j(n-h-2j,h-2i,k-i-j,l-j){Rk-i-i(l))(S>{Ri-j'tp) 
i=j 

is an explicit immersion of S^^"^'"^ <S S"'^^^^'^ into Res^" h^Sh^'^~^'^ ' 

Proof. Just note that by ([6]), only representations of the form S^~'^'^ S"'~^~^'^ can appear in 
the decomposition of Res|!" ^y,Sh^"^~^'^ irreducible representations. □ 

Lemma 2.5. Suppose again that < i < h/2, < j < {n - h) /2 and that (p G S''-'''-{A), 
ip € S"'~^~^'^{B). For < k < m < n and max{i — h + k,j} < I < m.m{k — i,n — h — j}, we 
have 



min{m—k+l,n—h—j} 



Rrn-k [{Rk~l-i4>) ® {Rl~M = d I 7-^/) ( Tr (Rm-w-iCl)) «> {Rw-j^l^) 

w=ma,x{l,i—h+m} 

Proof First of all, note that d*{(j)'®tp') = {d*(i)')®ii)' + (j)' (S>{d*tp') (analogous to (HD). Therefore, 
as in the Leibnitz rule of elementary calculus, we get 



Since RqRp = {^^'^)Rp+q, it follows that 



Rt [{Rqct>') ^ (i?pV')] = E ' + + ^) [{Rt-s+,^') ® {Rs+p^')] . (11) 

s=o ^ ^ /VP/ 

The formula in the statement is obtained by setting t = m — k,q = k — I — i,p = l— j and 
s = w — I (and taking into account [3] in Theorem 12.21 for the limits in the sum) . 

□ 
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Theorem 2.6. For < m,h < n, < k < min{n — m,m}, < i < h/2, < j < (n — h)/2, 
k-h<j-i<n-h-kandi + j<k, the map : S^'^'^A) (g> S'^~^~i'^{B) ^ M""™-'" given 
by setting, for(t)®^(^ S^-^^''{A) ® S'^-^'i'^ {B), 



min{n— /i— j,m— i} 

Tm{(t>®il^) = ^ Ek-i-j{n-h-2j,h-2i,m-i-j,w-j){Rm-w-i(p)^iRw-jip), 

to=max{j,i— h+m} 

is an explicit immersion of S^~'^'^ ® S'"'''^~^'^ into Res|" ^ \Rm-kS^~^'^\ - Moreover, 



x{n-h-j-k + i + l)k-i-j{h -i-k+ j + l)k-i-j{m - k + l)k-i-jX 

X (n — m — k + ^)k-i~j\\4'\\\.fh-iA(^y^^\\'4^\\\jn-h-j,j (^^y 

(12) 



n — 2i — 2j\ f n — 2i — 2j\ ^ n — i— j — k + 1 



RqTm — ( ]Tm+q- (13) 



Finally, 



Proof. First note that 

rmn{'m—k+l,n—h—j} min{n— /i— j,m— i} min{A;— 

E E = E E • (") 

«)=max{i,i— /i+m,} ■u)=max{j,i— /i+m} Z=max{jr,iii— m+A;} 

Then by composing the map in Corollary 12.41 with Rm-k (see [3] in Theorem I2.2p . we get 

k—i 

Rm-k ^ Ek-i-j{n - h - 2j,h - 2i,k - i - j,l - j){Rk~i-i4>) ® (Ri-jip) 

1=3 



min{n— fc— j;',m— i} 

E 

«)=max{j,i— /i+m} 



min{A;— j,ui} 

Elm — W — l\IW — 1 

i=max{j,w)— m+fc} 



k — I — i J \ l — j 



X 



X Ek^i^j{n - h - 2j,h - 2i,k - i - j,l - j) 



{R 

m~w—t 



min{n—h—j,m—i} 

= ^ Ek-i-j{n - h -2j,h -2i,m - i - j,w - i){Rm-w~i4>) ® {Rw-ji^) 

ui=max{j,i— h+m} 

where the first equality follows from Lemma 12.51 and ()14p and the second equality from ([2]) . 
This proves that is an immersion of S^'^^' ® S"--^~^'^ into Resf" ^ ^ \Rm-kS''~''^^] ■ 
Prom [2] in Theorem 12.21 we deduce that 
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\\{Rm-w-i<P) ^ {Rw-j'(p)\\'l,ln-m,,n — W - j J 

and then ()12p follows from the orthogonality relations ([3]). 

Finally, (fT3]) is a simple consequence of the identities Tm = Rm-kTk and RqRm-k = 

V q j^m—k+q- 

□ 

If X C {1, 2, . . . , n} and < t < |X|, we denote by <7t{X) the characteristic function of the 
set of all y C X, |y| = t. If max{0, m — /i} <w< min{n — /i, m}, we denote by am-w{-^)®(^'w{B) 
the characteristic function of the set of all Y € Vtm such that |y n A| = m — w and |y H S| = w 
(compare with ([6])). 

Corollary 2.7. /^)/ For < m,h < n and < k < min{n — m,m,n — h, h}, the space of Sn-h ^ 
Sh-invariant vectors in the representation Rm-kS"~^''' is spanned by the function <I>(n, h, m, k) = 
E!!^=max{o™i+m} ^k{n - h, h, m, w)am-w{A) ® a^{B). 

Proof. Take (p = o'o{A) and tp = ctq^B) (and therefore i = j = 0) in Theorem 12.61 □ 

In particular, the spherical functions of the Gelfand pair {Sn,Sn-m x Sm), normalized so 
that the coefficient of am{A)ao{B) is 1, are given by: (^_-^-^k(^n-m-k+i)k{m~k+i)k 
< A: < min{n — m, m}. 



3 The tree method for multidimensional Hahn polynomials 
3.1 Labeled trees and multidimensional Hahn polynomials 

We recall that a tree is a connected simple graph without loops or circuits. A rooted binary tree 
is a tree with a distinguished vertex a {the root) of degree 2 and all the remaining vertices of 
degree 3 or 1. The vertices of degree 1 are called the leaves, all the other vertices are called 
internal vertices, or branch points. In what follows, T is always a finite binary tree (identified 
with the set of its vertices). 

The ^-th level of a tree T, denoted by Ti, is formed by the vertices at distance / from the root. 
The height of T is the greatest L such that there exists a vertex in T at distance L from the 
root. If Q € 7/ is an internal vertex, then there exist exactly two vertices /?, 7 € Ti+i connected 
with a; they are called the sons of a, while a is the father of /3 and 7. We think of T as a planar 
tree, and therefore a has a left son and a right son. In the figure below, /3 is the left son and 7 
is the right son. 



a 




(5 7 

For a tree T, we denote by T' and T" the subtrees formed respectively by the left descen- 
dants and the right descendants of the root. We denote by a the root of T, and by (3 and 
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7 respectively its left and right son. Then /? is the root of T and 7 is the root of T". This 
decomposition will be the key for the following iterative procedure. 

Basic iterative procedures. 

• We define/prove something for the subtrees T and T", examining the particular cases in 
which /? or/and 7 is a leaf, and then we show how to pass to the entire T. 

• We define/prove something for the root a and then we show how to pass to its sons /? and 
7- 

Now we show how to use the second procedure to label the tree. We will use four type of 
labelings. 

The first is the composition labeling. Suppose that a = (oi, 02, ... , ah) is a composition of n. 
This means that ai, 02, . . . , a/i are positive integers and that ai + 02 + • • • + a/i = n; ai,a2, ■ ■ ■ ,ah 
are the parts of a. We will write a Ih n to denote that a is a composition of n. Suppose also that 
T has h leaves. We denote by T(a) the labeled tree obtained in the following recursive way. The 
label of a is (oi, 02, ... , Oh)- If T' has t leaves, then the label of its root /3 is a' = (oi, 02, . . . , at), 
while the label of 7 (the root of T") is a" = {at+i,at+2, ■ ■ ■ , o-h)- Note that given a and T, the 
labeling is unique. We give two examples with h = A. 




In what follows, to simplify notation, we set Of- = ai + 02 + ■ ■ ■ + at (and therefore a' Ih 
and a" Ih n — a^). 

The second is the variables labeling. It is defined simply by taking a set of variables indicized 
by the internal vertices of T. The resulting labeled tree is denote by 7^(w), where w = {wt)t£T 
is a vector of variables (and w-r is the label of r G T; often, the index r will be omitted). The 
leaves do not have labels. 

The third is the spaces labeling. It is defined by choosing a label < m < n for the root a; 
if w is the variable associated to the root, then m — w \s the label associated to (3 and w is the 
label associated to 7; the leaves are not labeled. The resulting labeled tree is denoted by %{m). 

The fourth is the representations labeling. It is based on Corollary 12. 4[ The label of a leaf is 
zero. Suppose that k is the label of a, i is the label of /3 and j is the label of 7. 
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• If /3 and 7 are both internal we must have 



< A; < min{n — m, m} (15) 

and 



0<i<aj2, 0<j < (n-aJA ^^^^ 

k — a^<j — i<n — a^j. — k, i + j < k. 

Suppose /3 is a leaf (resp. 7 is a leaf) and 7 is internal (resp. P is internal). Now t = 1, 
i = and k,j must satisfy the conditions in (llSp and ()16p . with i = (resp. t = h — 1, 
j = and /c, i must satisfy those conditions with j = 0). 



• Suppose that (3 and 7 are both leaves: now h = 2 and (fTSll . (fTBl) reduce to < fc < 
min{ai, 02}. 

We may also say that (fTSj) . (fTBI) must be satisfied and that we must have i = (resp. j = 0) 
if l3 (resp. 7) is a leaf. The resulting labeled tree will be denoted by ^^(k). 

Clearly, the spaces labeling depends on n and the variable labeling, while the representation 
labeling depends on both the composition and the spaces labeling. 

Once we have labeled T as above, then T'(a), T"(a), . . . will denote the subtrees T' and T" 
with the labeling inherited by T. 

Now we define a set of multidimensional Hahn polynomials associated to the tree T with the 
labelings defined above. We keep all the preceding notation; the definition is recursive. 

• For 



max{j, i — at + 1^ w < min{n — Oj. — j,rn — i}, (17) 

we set 



^r,{k)(^(a), m, 7;(w)) = Ek_^i_^j{n - at - 2j,at - 2i,m - i - j,w - j)x 

X ^T/(k)(T'(a),m - wXM)ET,"{k){T"ia),wX'M) 

• If /3 is a leaf, then w must satisfy p!7|) with i = 0, and we set Eq-^^y^{T' {a),ni — w, 7^'(w)) = 
1. 

• If 7 is a leaf, then w must satisfy (fT7|) with j = 0, and we set E-j-n(y^{T" {a^^w^ 7^"(w)) = 1. 

• If /? and 7 are both leaves, w must satisfy pT]) with i,j = 0, and 

-E^r,(k)('^(a),m,7;(vi^)) = Ek{a2,ai,'m,w). 
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Remark 3.1. The conditions (|15p . (|16p and (jl7p are imposed by Corollary 12.41 and Theorem 
12.61 and agree with ([1]). Moreover, when (jl7p is satisfied, we have automatically < i < 
minjoj — m + w,m — w} and < j < min{n — a± — w, to}, that is we do not need to impose 
the analog of (jlSp to In other words, once we have imposed (jl7p . the labels in ^^(k) must 
satisfy only the conditions obtained by iterating (I16p . while the variables in 7J,(w) must satisfy 
only the conditions obtained by iterating (jl7p . 

Example 3.2. Suppose that /i = 4 and take the tree and the labelings below. 




ai a2 as 04 








Then, for 0<m<n = ai+a2 + a3 + a4,0<A:< min{n — oi, n — 02, n — 03, 71 — 04, m,n — Tn}, 
< i < min{ai, 02}, < j < min{a3, 04}, k — ai — a2<j — i<a^ + a4, — k and i + j < k, the 
associated Hahn polynomials are 

-E'r,(k)C^(a), m, 7;(w)) = Ek-i^j{a3 + 04 - 2j, ai + a2 - 2i,m - i - j,w - j) x 

X £'4(02, fli, m — ty, v)Ej{a4, a^iW, u) (18) 

defined for max{j, i — oi — 02 + 7Ti} <w< minjoa + 04 — j, m — -i}, max{0, m — oi — tn} < v < 
min{a2, m — w} and max{0, w — 03} < t( < min{a4, w}. The conditions /c < n — Ur, r = 1,2,3,4, 
come from the conditions on i,j and agree with the Young rule for the symmetric group. 

Example 3.3. Now we take a general composition a = (oi, 02, . . . , a/i) but a special kind of 
tree. To simplify notation, we set wq = m. 
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We introduce a specific notation in this example: for r = 1, 2, . . . , /i — 1 we set 

ttr = min{ar+i + 0^+2 H + a/i, «r + «r+2 + ar+3 + • • • + O/i! • • • , «r+l + ar+2 + 1- O/i-l}- 

For < wq < n, < ki < min{n — wo,wo,ai}, 

max{0, kr-i — a-r-i} < kr < minjor. + • • • + o/i — fe^-i, kr-i,ar}, r = 2, 3, . . . , /i — 2, 

and max{0, A;/i__2 - a/1-2} < ^/i-i < min{a/i_i + ah - kh^2, kh-2,ah} (we also set /c/^ = 0) the 
associated Hahn polynomials are 

h-l 

E%(]<.)i'^i^),wo,%{.'^)) = JJ Ek^_kr+i{ar+i H \-ah- 2kr+l,ar,Wr-l - kr+l,Wr - kr+l) 

r=l 

defined for 

nia,x{kr+i,Wr-i — flr} < Wr < minjor+i + • • • + a/i — kr+l, Wj—i}, r = 1,2, . . . , h — 1. 
The condition k^-i < a/i-i comes form Corollarv l2 . 71 and forces kr < cir, r = h—2, h—3, . . . , 1. 
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Remark 3.4. The preceding example gives the multidimensional Hahn polynomials in [15] and, 
as a special case, those in [9j. 

Indeed, Dunkl's function (— l)'^'^™'6'(ni, M2, ^3; a, b, c) is obtained by setting h = 3, (ai, 02, as) = 
{c,a,b),ki = r,k2 = m and m = M, with the variables wi = ui + U2 and u;2 = U3 (one has 
just to apply the symmetry relation (3.2) in [B] (see ([55]) in section ESI of the present paper) to 
Er-m{- • • ), and this gives the factor (—I)''"™). 

More generally, in the Karlin and McGregor notation ([15j. p. 277) the function 



Wh-l-Wh, Wh-2-Wh-l, WQ-Wl 

\kh-i, kh-2-kh-i, ki-k2 

-Qh-i-l, -ah-2 -I, -ai-1 

is a multiple of our E-j-^^^i^-^iT^a^^wojT^iw)) in the preceding example (one has to use the 
formulas in [B], p. 631, keeping into account that Qm{x', a, P, N) in [B] is equal to Qm{x; a, P, N + 
1) in [l5]). Tratnik [28] gave a multi variable version of the Askey- Wilson polynomials that 
included the Karlin-McGregor multi variable Hahn polynomials (and the (/-analog of Tratnik 
construction is given by Gaper and Rahman in [H]); it is natural to ask if the construction in 
the present paper may be generalized to the Tratnik (or Gasper-Rahman) setting (the g-analog 
of [9] is in [8]). Other multidimensional g-Hahn polynomials are in |20j . 

3.2 Sa — Sm X Sn-m intertwining functions 

Now we define a set of Sa-invariant functions in M^~''^''^. 

• The function <I>(T(a), m, 7^(k)) e ]\^n-m,m -g (^gg^ed by setting, iteratively. 



min{n— flj— j,m— i} 

$(T(a),m,7;(k)) = ^ Ek-i-j{n - a^. - 2j,at - 2i,m - i - j,w - j)x 

to=max{j,i— a^+m} 

X $(r'(a),m-u;,r/(k)) (g)^>(r"(a),u;,r/'(k)). (19) 

• If /? is a leaf (resp. 7 is a leaf) then i = and t = 1 (resp. j = and t = h — 1) and we 
set $(r'(a),m-u;,T/(k)) = a„(^i) (resp. $(T"(a), u;, T/'(k)) = a^(A)). 

In particular, if f3 and 7 are both leaves we have 

min{n— ai ,m} 

$(T(a),m,7;(k)) = ^ £^^(02, Ol, "i, 'lf)crm-i«(^l) <X) O■«;(^2)■ 

UI=max{0,m— ai } 

Lemma 3.5. For k < r < n — k, we have 

i?,_^<I>(T(a),m,T,(k)) = T^^M ci>(T(a), r, T,(k)) 
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Proof. The proof is by iteration. We limit ourselves to examine the case in which both (3 and 7 
are branch points. Applying (jlip . with t = r — m, p = q = and s = v — w, we get 



min{n—a^—j,m—i} mm{n—a^—j,r—rn+w} 

Rr-m^{T{a.),m,%.(k)) = ^ ^ Ek-i-j{n-at-2j,at-2i,m-i-j,w-j)x 

ui=max{jr',i— flj+m} t)=max{«),r— a^+i} 

X [i?,_„_,+^$(T'(a),m-«;,T;(k))] [i?,_^^>(T"(a), ti;, T;'(k))] 



min{n— Oj— j.r— j} 

E 

■(;=max{j,r— a^+i} 



min{ii,m— i} 

^ Ek-i-j{n - at, at - 2i,m - i - j,w - j) x 

.«)=max{j,D— r+m} 



r — V — I \ ( V — J 



$(T'(a), r - w; - s, T/(k)) ® $(T"(a), + s, T/'(k)) 

m — w — ij \w — J ' 

;-_;)*(T(a),.,T,(k)). 

where the second equahty follows from the induction hypothesis and last from ([2]). 



□ 



Corollary 3.6. In the notation of Theorem \2.(k we have: 



$(r(a)),m,r,(k))=r„[cI>(T'(a),i,T/(k))0cI>(T"(a),j, T/'(k))] 

Let L be the height of T. To the labeled tree T(a), we associate a chain of subgroups 
= Sn > Ki > ■ ■ ■ > Ki-i > Ki = Sa defined as follows. Let L' and L" be respectively 
the height of T' and T" and suppose that Kq = Sg.^ > K[ > ■ ■ ■ > K'^, = Sa^ x • • • x Sat 
and Kq = Sn-a^ > K'l > • • • > K'l/i = Sat+i x • • • x Sa^ are the respective chains. Clearly, 
L-1 = max{L', L"} and if L' < L - 1 (or L" < L - 1) we set K'^,^-^ = ... = K'^ -^ := K'^, 
(resp. K'[„_^_i = ■ ■ ■ = K'[_^ := K'[„). Then we define the chain of T(a) by setting 

Kq = Sn and Kj = K'j_^ x K'J_^, for j = 1, 2, . . . , L. 

For instance, the chain associated to Example l3.2l is 5„ > Sa-i+a2 ^ 5*03+04 ^ x x x 
Sa^, while the chain associated to Example l3.3l is 5„ > Sai x S'^jh hah ^ "5*01 x •S'aj x Sa^-i > 

■ ■ ■ ^ 'S'q, J x Sfj^ X • • • X Saf^ . 

Let A; be as in (jl6p . We define a chain of irreducible representations Wo = S"'^^'^ 5 
Wi 5 • • • 2 W^L = ^("i) <8) • • • S'("'^) associated to the labelings T(a) and 7;(k); the subspace 
Wj is an irreducible representation oi Kj, j = 0,1,2, L. Suppose that Wq = D W{ 3 

■ • • D and VF^' = ^'^-^^i-^J' 2 W^i" 2 • • • 5 Ty['„i are the chains associated to T' and T". 
Then the chain associated to T is 

Wo = 5"-'='*= and W,- = W^_^ j = 1, 2, . . . , L. (20) 

Since the decomposition in (j2.4p is multiplicity free, the chain of representations is uniquely 
determined by the tree and its labelings. In other words, the restriction Res^"S'"'~'^''^ does not 
decompose multiplicity free, but using the transitivity of restriction, we may write 
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Resf^S'^-'^'^ = ResJ;;-^ • • • Res^iRes;^J5'^-^'^ (21) 

Then Res^'^'^""'^''^ decomposes multiplicity free and we choose the unique subspace isomor- 
phic to (8> S"'~-^~^'^ , and we iterate this procedure. Since the leaves of ^^(k) are labeled 
with zeros, this implies that Wl is the trivial representation of Sa^ x • • • x 

For instance, the chain of representations associated to Example 13.21 is S^~^'^ ^ gai+a2-t,i ^ 
ga3+a4-j,j > 5'(ai)05(a2)05'(a3)(g,5(a4)^ ^j^jlg ^-j^g g^ain associated to Example[33]is S""-^^^^^ ^ 

(g) ga.2~\ Vah-k2M 3 (g) 5(^2) ^ ^"laH hafe-fcs.fca 3 ... 3 S^^i) (g) 5(^2) (g) • • • (g) S^'^'^\ 

Denote by [S^"^) (g • • • (g 'S'''"''^]^^^!^) the subspace obtained by mean of the particular 
labeling 7^(k) (while '7"(a) is constant). Then 



(g ^^'''^^ 



A: fixed 



r,(k) 



(22) 



is an orthogonal decomposition of the S^"''^^®- ■ •(gS'^'^^^-isotypic component of Res|^ x---y.Sa 
(the sum is over all labelings of T with k as label of the root). Indeed, different choices of the 
parameters of the labeling give rise to orthogonal subspaces (at the first stage of (pT|) 

in which they differ); and varying the parameters we obtain a every possible subspace in the 
intermediate decompositions. 

Theorem 3.7. The Sa—{Sh>< Sn-h)-iiT'Variant function $(T(a), m, 7^(k)) belongs to the subspace 
Rm-k (g • • • (g 5^"'^^]g;,(k) Contained in C M""™'™. 

Proof. The proof is again by iteration. Suppose that it is true for $(T'(a),m — w, %'{]<)) and 
^{T" {a),w, Tj(k)). Then one can use Lemma FS.SI and Theorem l2.6l to deduce that <I>(T(a), m, '?r(k)) 
belongs to the subspace 5^-*'* (g contained in the restriction of Rm-kS'^~^'^ to Sn-a^ x 

Corollary 3.8. The set {$(T(a), m, 7^(k)) : ^^^(k) is a rep. labeling } is an orthogonal basis 
for the space of Sa-invariant functions in M^' 



fn—m,m 



The norm of <I>(T(a), m, 7^(k)) will be computed in the next section. 



3.3 The Sa — Sm x Sn-m-intertwining functions as Hahn polynomials 

Now we want to express the functions <I>(T(a), m, 7^(k)) in terms of the Hahn polynomials 
E'j-^(^\i-j{T{3L),m,%{-w)). Set Aj = {ai + • • • + aj_i + 1, . . . , ai + • • • + aj} and suppose that 
Sa^ is the stabilizer of Aj, j = 1,2, ... , h. If xi,X2, ■ ■ ■ ,Xh are nonnegative integers satisfying 
xi + X2 + • • • + Xfi = m, we denote by axi{Ai) g) (T^a 'Si {A2) (8> • • • ax^{Ah) the characteristic 
function of the set of all B £ such that |-B fl = Xj,j = 1,2, ... ,h; compare with the 
notation in Corollarv 12.71 Since the functions cj^^(^i) (g ax2{A2) ig • • • (g a^^iAh) form a basis 
for the 5a-invariant functions in j^f"-*"."*^ each <I>(T(a), m, 7^(k)) must be a linear combination 
of these functions. The following Lemma is obvious. 

Lemma 3.9. We have 
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$(T (a), m, Z(k)) = ^r,(k)(^ (a), m, %{w))a^^ (Ai) ® ^7^2(^2) «) • • • fx^ JA,,) 

where the suTYi is ovev all possible value of the vuTiubles Z7i'7^(w) ajid the coefficiefits xi, . . . ^Xfi 
must have the following values: 

• if Oj is the label of a leaf which is the left son of a vertex with y as space label and z as 
variable label, then xj = y — z; 

• if Oj is the label of a leaf which is the right son of a vertex with z as variable label, then 
Xj = z. 

For instance, in Example 13.21 the invariant functions have the expression: 



Ek-i-j{a3 + 04 - 2j,ai + a2 - 2i,m - i - j,w - j)Ei{a2,ai,m - w,v)x 

w,u,v 

X Ej{a4,a3,w,u)am-w-viAi) (S> 0-^,(^2) ^ cr^-uiAs) (S> o-„(^4), 
while in Example 13.31 they are 

/.I \ 

1) I ^wo—w\ 

Wl,...,Wh_l \j = l J 

(^2)0 {Ah-2) ® cr^h-iiAh) 

The polynomials E-j-^^y^{T{a),m^ 7^(w)) satisfy a number of properties that may be deduced 
from their group theoretic interpretation or from the identities in section [27T] (and those in [5]). 
We give two examples. 

Suppose that T is a tree, with the labelings used in the previous sections. Suppose that 
7^(v) is another variable labeling and ^(r) another space labeling of the same tree. We define 
a set of binomial coefficients by setting 



if (3 and 7 are internal; 



if /? is a leaf, is replaced by {^JJ; if 7 is a leaf, (^J^Sw)) i^ replaced by Q. 

Then from Lemma 13.51 (alternatively, from ([2])) one can easily deduce the multiplication 
formula: 

E ^-.*'<^<=''-'"'^-(~))(™.?(w)) = (;:t.)^T.M(T(a),.,7;(v)). 

7^ (w) 

Now suppose that 7^(k) and ^^(k) are two representation labelings for T. Clearly, 
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{^{T (a), m, Tr(k)), ^{T (a), m, T,(k))>^„_,^ = 5r.(k),r.(k) (a), T,(k))||^„_.^ . 

Moreover, the norm ||<^(T(a), m, 7^(k))||^^„_„_^ may be easily computed by iteration using 
W) and Corollary [Ml 

||cI>(T(a),m,T,(k))||^,„__ = " '^V" " " ^ ' H 

\m — I— jj\k — I— jj ^J- 

where the product is over all internal points uj of T and 



L = : — ; — TTZrVi ('^^+1 \-at-z-x + y + Ij^-y-z'X 



ar-\ \-at — x-y — z + 1 

ar + • ■ ■ + at — 2x + 1 

x(arH \-as-y-x + z + - x + l)^_j^_^(ar H \- at - q - x + l)x-y~ 



if (a^, ar+i, . . . , Of) is the composition label of lo, q and x respectively its space and represen- 
tation label, (ur, . . . , as) and (og+i, . . . , at, ) the composition labels of his sons (left and right) 
and y, z their representation labels. Define a weight by setting, inductively. 



VF(T(a),m,T,(k),Tv(w)) 



a^ — 2i \ / n — a^ — 2j 
m — w — i J \ w — j 



and W{T{a),m-w,Tj{k),T^{w)) = 1 (resp. W{T"{a),w,T^"ik),T^'{w)) = 1) if /3 is a leaf 
(resp. 7 is a leaf). Then the orthogonality relations are: 

Er^^^){T{R),m, r,(w))i?^^(j^)(r(a), m, %{^))W{T{a),m, %{k),%{w)) 



■m.m 



3.4 On Frobenius reciprocity and Mackey's Lemma for a permutation rep- 
resentation 

In this subsection, we establish some technical facts on permutation and induced representations. 
We begin with a closer look at Frobenius reciprocity. Let G be a finite group and K a subgroup 
of G. If V is any G— representation, we denote by the subspace of ET— invariant vectors in 
V. If X = G/K and xq G ^ is the point stabilized by K, then the space L{X) may be identified 
with the space of right JC-invariant functions on G via the map 

L{X) L{G) 

(23) 

where f{g) = f{gxo) for any g e G. 
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Suppose now that (p, V) is irreducible and unitary and that is non-trivial. For any 
V € , define the linear map Ty : V ^ L{X) by setting 

{T,uw){gxo) = {w,p{g)v)v, (24) 

for any w & V and g G. Clearly, TyW € L(X) is well defined because v is i^-invariant. 
Moreover, it is easy to see that T„ G KomdV, L{X)). 

In particular, one can easily gets Frobenius reciprocity for permutation representations: the 
map 

^RomG{V,L{X)) ^^^^ 
V I — > Ty 

is an anti linear isomorphism between the vector spaces and HomG'(y, L{X)) (and there- 
fore the multiplicity of V in L{X) is equal to the dimension of [3]). 

In the following Lemma, we compute the invariant vector associated to the restriction of the 
intertwining operator Rm-k to S"'~^'^; see Theorem I2.2i We use Corollarv 12.71 ^{n,m,k,k) is 
the Sn-m X S'm-invariant vector in S'^~^'^. 



Lemma 3.10. In the anti linear isomorphism (j25p . we have: 
Rm-k\sr^-k,k = Ty, where 

-— -^{n,m,k,k). 



{n-2k + 2)k-ik\ 



Proof. Set y = {1, 2, . . . , m}, Z = {m + 1, m + 2, . . . , n} and suppose that Sn-m x Sm is the 
stabilizer of Y. First of all, we prove that on the whole M^~^'^ we have Rm-k = Tu, with 
u = (Tk{Y) (g) ao{Z). Indeed, \i F eVtk then ([Ml) yields 



{Tu5F){gY) = {5F.g[<yk{Y)®ao{Z)]) 
= {Rm-kSF){gY) 



liiF CgY 
otherwise 



for every g G 5^. Then it suffices to compute the projection of ak{Y) (g) ao{Z) onto 5" 
since the dimension of the Sfi—m 

X Sm-invariant vectors is equal to 1 (and is spanned by 
$(n, m, /c, /c)), such projection is given by 



<^(n, m, k, k) 



|<I>(n,m, fe, k)\Y, 



-(o-A:(^) ® CToiZ), ^{n, m, k, k)) Mr^-k,k 



{n-2k + 2)kk\ 



<I>(n, m. A;, A;). 



□ 
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By combining the preceding Theorem with Lemma [3. 51 we get the expression of $(T(a), m, 7^(k)) 
as a matrix coefficient. 



Corollary 3.11. // r/ = $(T(a), m, 7;(k)) then 



Now we examine the behavior of the map (j24p with respect to transitivity of induction. 
Suppose that that H is another group with K < H < G. Set Y = G/H and Z = H/K, and 
let Ho € Y he the point stabilized by H in Y. Identify Z with the subset {hxQ : h € H} of X, 
so that xq is also the point stabilized by K in Z. For every y € Y choose Sy £ G such that 
SyUo = y- Then we have G = ]J SyH, that is {sy : y € y} is a transversal for in G; in 

particular 

X = ]J (26) 

Transitivity of induction for a permutation representation may be made more explicit by 
saying that 

LiX) ^ IndgL(Z) = L{syZ), (27) 

as follows immediately from (|26p . Let (/O, T^) be again a unitary, irreducible representation 
of G and an ff-invariant, irreducible subspace of V. That is, {a, W) is an irreducible repre- 
sentation in Res^y and a{h)w = p{h)w for all w G and h € H. If wq G W is /C-invariant, 
then, by mean of (f24|) . we can form two distinct intertwining operators: Swq '■ W — > L{Z) (that 
commutes with H) and T^jq : V L{X) (that commutes with G). From the definition of 
induced representation [2T] and (i27l) . Ind^S^^VF is a well defined subspace of L{X), namely 

lnd%S^,W = ^SyS^,W (28) 



Lemma 3.12. The operator T^q intertwines V with the subspace Ind^5^gl^, that is T^qF C 



Proof. Denote by Pw : V ^ W the orthogonal projection onto W . Suppose that x £ X and 
X = Syhxo, with y € y and h £ H. By (p^ . for any v £ V we have 

(ru,ot7)(j;) =(r^oi;)(sj//ij;o) = {v, p{sy)p{h)wo)v 

= {Pwp{sy^)v,a{h)wo)Y = [SyjoPwp{sy^)v] {hxo) 
= {sy [SwoPwp{Sy^)v] } (x) (because x = Syhxo), 

that is, T^qV G SyS^gW, and by ([25]) this shows that T^gf G Ind^^^gW^. □ 
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In the last part of this subsection, we show how to apply Mackey's lemma in our setting. 
Assume all the preceding notation and suppose that J is another subgroup of G. Suppose that 
T is a set of representatives for the J — H double cosets of G, that is G = JtH. Set 

Q,t = {gtyo : g G J}. Then Y = U^g'p is the decompositions of Y into J orbits. Let Jt be the 
stabilizer of tyQ G VL^ in J. Clearly, Jt is also the group of all 5 E J such that gtZ = tZ. Then 
Mackey's Lemma |21) in our context coincides with: 

Res^f = Res^IndgL(Z) = Ind;^^L(tZ) = L{Qt) (29) 
where @t = {gtz : z & Z,g £ J}. 
3.5 The Sn~m x iSm-invariant functions in M'^ 

Let a = (oi, . . . , ah) be again a fixed composition of n. We denote by Qg, the S'n-homogeneous 
space of all (Bi, B2, ■ ■ ■ , Bh) such that: Bj is a j-subset of {1,2,..., n} and Bi r\ Bj = % for 
i 7^ j, (and therefore Si U • • • U = {1, 2, . . . , n}). Following the standard notation, we denote 
by the Sn-permutation module Liytg). If x = (xi, . . . is another /i-parts composition, 
we will write x < a when xj < aj,j = 1,2, ... ,h. If x < a, x Ih / < n and F is an /-subset of 
{1,2,..., n}, we can construct an fix space using the subsets of F, and we denote it by r2x(-^); 
the corresponding permutation module will be denoted by M^{F). 

Set again y = {1, 2, . . . , m} and Z = {m + 1, m + 2, . . . , n} and suppose that Sn-m x Sm is 
the stabilizer of y G Jim- The decomposition of into Sn-m x Sm orbits is: 

^^a = U [^-(^) ^ ^a-x(^)] , (30) 

xlhm 
x<a 

where a-x = {ai - xi, . . . ,ah- Xh) and VLy^{Y) x r2a-x(-Z') = {(-61,-62, • • • , Bh) G : \Bj n 
y| = Xj,j = 1,2, ... ,K}. The consequent decomposition of lie,s^g^^_^^g^M^ into permutation 
modules is 

= [M''(y) ® M^-^(Z)] . 

xlhm 
x<a 

For X Ih m, X < a, we denote by crx(y) the function in M^{Y) which is constant and iden- 
tically 1. Then the tensor product o"x(y) ® aa-:x.{Z) is the characteristic function of the orbit 

n^iY) X 17a-x(^). 

If r/ G then (j23p yields the function defined by setting rj(g) = rj{gY); similarly, 

if 61 G M"", then 0{g) = 9{gAi, . . .,gAh). If G M""'"'"' is 5a-invariant, then rj is also left 
^a — {Sn-m X 5'm)-invariant; then we can define r/'* G M'^ by requiring that rj^g^^) = rj^g). 
In other words, the change of variable g 1-^ g~^ establishes an isomorphism between the vector 
spaces of S^ — {Sn-m x 5'rrt)-invariant functions and {Sn-m x 5m) — Sa-invariant functions on 
Sn, and on the homogeneous spaces this isomorphism becomes (^M"'~"^'"^)^'^ — > (^j^^a-^Sn-mxSm ^ 
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If r/ = • • (g)C7^^(A) and 9 = axi,...,XhO^)®(^aT,-xi,...,ah-Xh{Z) then = 9. Indeed, 

77 is the characteristic function of all g ^ Sn such that \gY T\ Aj \ = Xj,j = 1,2, ... ,h, while 9 is 
the characteristic function of all g G Sn such that \Y n gAj\ = Xj,j = 1,2, ... ,h, and therefore 
rj{g~^) = 9{g). As an immediate consequence, we get: 

Lemma 3.13. Under the map rj 1— > r/^, the image of ^{'T{ai),m,%-(k)) is the function 

^{T{a),m,Z{k)) = Er,ik){T{a),m,%{w))a^{Y)0a^.^{Y) (31) 
rv(w) 

where the sum over is as in Lemma \3.9\ (with the same conditions onx = (xi, . . . , Xh) )■ 

This Lemma gives immediately some of the basic properties of the ^ functions. 
Corollary 3.14. The set 

{•^{T{a),m,Tj.{k)) : %.(k) is a rep. labeling } 
is an orthogonal basis for the space of Sn-m x Sm-invariant functions in M^. 
Corollary 3.15. 

||*(r(a),m,r,(k))||i,. = ^ll^^l_:^||$(r(a),m,Tr(k))||i,™. 

(n — m)\m\ 

We need a little more work to establish the properties of the ^ functions corresponding to 
([19]) and Theorem 13. 7i If x = (xi, . . . ,Xh) is an h-parts composition and 1 < t < h is fixed, as 
in section 13.11 we set x' = {xi,X2, . . . ,Xt) and x" = {xt+i, Xt+2, ■ ■ ■ , Xh)- In the present setting, 
(ETI) is: 



W = Ind-^ 



[Af"'(F)®M'^"(F^)] ; (32) 



now Y is replaced by i X by fia and Z by 0,^' x ^a" ■ If we set 

= {{Bi, . . . ,Bh) e : \{BiU ■ ■ ■ U Bt) nY\ = m - w} 
then Mackey's Lemma (|29p in this setting is just 



Resf" Af^ 



mm{m,n—a^} m'm{m,n—a^} 

0" L(ej= 0" M-'(F)®M-"(F 

to=min{0,m— a^} w=min{0,m— ct^} ^Gf^at- 



Fny|=m— 1 



(33) 

Now we use Frobenius reciprocity to construct a sequence of irreducible representations from 
the sequence ([2U|) . The reciprocal of ([2T]) is 

M*^ = Indf oindg • • • IndJ;;-^ Is'-"'^ • • • S^"'^^ 

Then we can define a sequence Vq; ^ij • • • ) where V^- is an irreducible representations of Kj 
isomorphic to Wj, Vl is ® ■ ■ ■(^S^"''^^ and, by backward induction (j = L — l,L — 2,... ,1,0), 
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K 

Vj is the unique subspace in Ind^^.^_^ V^+i isomorphic to Wj (it exists and is unique because the 
multiphcity of VFj+i in Res^-^^^Wj is one). Then Vq is a subspace of isomorphic to S^~^'^, 
and we denote it by [S'^'^''^^^- (VlY Clearly, 



r.(k) 

k fixed 



is an orthogonal decomposition of the 5" '^''^-isotypic component in M^, reciprocal to (j22p . 

Lemma 3.16. If v = ^(T{a),k,%-(k)) and is the corresponding intertwining operator as in 

] r,(k) • 



then T^S"^-^^^ = [^"-'^''^^ 



Proof. Let Tj : Wj — > Lid^^VF^ be the intertwining operator associated to the S'a-invariant 
vector V (which belongs to each Wj by Theorem 13. 7p : in particular, Tq = T^,. Then a repeated 
application of Lemma [3.121 yields: if TjWj = Vj then Tj^iWj-i C Ind^^.^^l^-, and therefore 
Tj_iWj^i = Vj-i. This ends with T^S""-^^^ = Vq. □ 

Then from Corollary 13.111 and Lemma 13.131 we get immediately 



Corollary 3.17. The Sn^m x Sm-invariant vector ^{T{a),m,Tr{k.)) belongs to [5" ^'^] 



Tr{k)- 



In the last part of this section, we establish the inductive way to describe the ^ functions 
(reciprocal to (fT9l) ). For any F G 0^^, with |Fny| = m—w, we can take '$(T'{a),m—w,Tj.'(k)) G 
W'{F) and ^{T"{a.),w,%."{k)) G M'^"{F'^). We denote by 

[*(T'(a), m - w, T/(k)) ^{r'ia),w, T/'(k))] ^ 
their tensor product and we set: 

E{T{a),w,Z{k)) = J2 [^(T'(a),m-t/;,T/(k))®*(T"(a),w;,T/'(k))]p. 

FGna^-\FnY\=m-w 

In the notation of (|33p . we have H(T(a), if, 7^(k)) G 0^;. 
Proposition 3.18. For the function, we have the following iterative formula: 

min{n—a^—j,m—i} 

*(T(a),m,7;(k)) = ^ Ek_i_j{n - - 2j,at - 2i,m - i - j,w - j)x 

w=max{j,i—a^+Tn} 

xEiT{a),w,Z{k)). 

Proof. Keeping into account the decomposition (|33p . this formula is just a rearrangement of 
(ED. □ 
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3.6 Regge's symmetries for the Hahn polynomials 

In this section, we want to show how the tools developed in the setting of the multidimensional 
Hahn polynomials may be also used to sketch a group theoretic proof/interpretation of the Regge 
symmetries for the one variable Hahn polynomials. 

Lemma 3.19. The Hahn polynomials satisfy the following symmetry relations: 

Em{a, b, c, x) = {-l)"'Em{b, a,c,c- x), (35) 



Em{a,b,C,x) = (-1)""^- — — -r- -r— -r Ea-m{a, a + b -2m, a + b- 771- C + X , x) , 

[a + b — c — m)\{c — m)!(a — m)\ 

(36) 

Em{a, b, c,x) = Era{a + b - c,c,b,b - c + x). (37) 

Proof. Consider the Hahn polynomials (and the associated intertwining functions) in the Ex- 
ample (HTSl Suppose h = 1 and take 2n in place of n. Let (ai, 02, 03) be a composition of 2n with 
0-1,02,03 < n and set ki = wq = n. These conditions force k2 = n — ai. That is, 5^"^^ (8) 5"'""°^ 
is the unique subrepresentation of Res^^^"^^ _^ ^n,n q£ form S^"'^^ ® go-2+a3-j,j (Corollary 

I2.4p . A consequence of this fact is that the dimension of the space of 5^^ x x Sag-invariant 
vectors in 5"'" is one (it is not zero because n > ai, 02,03). Moreover, from the results in section 
13.31 we know that this space of invariant vectors is spanned by the function 

„ min{a3,iu} 

$1(01,02,03)= (-l)"-"^ai!(n-w)!(tt;-n-hai)!x 

w=n-ai u=miix{0,w-a2} ^ ' 

X En (A2) 0a„(^3)• 

(we have also used the identity E'aj (ai, ai, ai, n-l-ai) = (— l)"~"'ai!(n— i(;)!(i(;— n+ai)!; see 
dl])). Clearly, for every permutation vr of {1, 2, 3}, $1(0,^(1), a7r(2)j 071(3)) is still an Sa^^ x x Sa^- 
invariant vector in 5"'". Therefore there exists a complex number A^r such that ^>i(ai, 02, 03) = 
A7r^>i(a^(i),a^(2),a^(3)). Noting that 

• the coefficient of (Ja^{Ai)®an-aAM)®cFQ{A3) in ([Ml) is (-l)"(ai!)^(n-ai)!(n-a2-hl)„_ai , 

• the coefficient oi aa^{Ai)<^ao{A2)®an-aAA3) in ([Ml) is (-I)"! (ai!)^(n-ai)!(n-a3+l)„_ai , 

• the coefficient of an-a2{M) ® 0-^2(^2) ® 0-0(^3) in ([38]) is {-lY^'^^'^aiW-o^l, 

one can easily conclude that A(23) = (— l)"~"i and A(i2) = (— 1)""°^ 021(^-02)! ' '^'^i^ yields 
and 



To get p7p . first note that if 77 = ^{n,h,m,k) and iji = ^{n,m,h,k) then from Corollary 
([3TT]) we get 



[n — Ik + 2)kk\ 
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and 



.l)k 



[n — 2k + zjfc/c! 



and therefore 



v{g-^) = mi9)- (39) 

On the other hand, if A, B are as in section 12.31 and Y, Z as in Lemma 13.101 then 



V{g) = Ek{n-h,h,m,\gY ^B\) and r]i{g) = Ek{n - m,m,h,\gAr^ Z\). (40) 

Indeed, am-w[A) (g) (Tw(B) is the characteristic function of the set {F G ■ \F f] B\ = w}. 
Then ([37]) fohows from ([39]) and (gOD, noting that I5("^yn-B[ = m-\YngA\ = m-h+\gAriZ\. □ 

To give a more symmetric formulation of Lemma l3.19t we introduce a Regge's array notation 
for the Hahn polynomials. Set 



an 


ai2 


Ol3 




021 


022 


023 


=( 




032 


033 





03i!a32!ai3!o23! 



a22!a2i!oii!ai2!a33!(oii + 012 + 013)! 



1/2 



X -£'033(011 + 012,021 + 022,012 + 022,012). 

In the array, the sum of each row and the column has the same value L, and this fact 
determine 013,023,031,032,033. The array is invariant under transposition and under even per- 
mutations of the rows (or of the columns), while is multiplied by (—1)^ under an odd permutation 
of the rows (or of the columns). Indeed, the invariance under transposition is equivalent to (j37p . 
exchanging the first row with the second row we obtain (j35p . and exchanging the second row 
with the third row we obtain (I36D. 



Remark 3.20. In Lemma 13.191 we have used two different invariant vectors to get all the 
symmetry relations for the Hahn polynomials. We suppose that developing the techniques in 
[22] (namely the theory of Sai x x Sa^ — Sb^ x Sb^ x -invariant vectors on Sn) one might 
get a single vector whose invariance properties yield all the Regge symmetries. This approach 
might be connected with |10j . 



4 Hahn polynomials and Clebsch-Gordan coefficients 
4.1 Clebsch-Gordan coefRcients 

Now we recall some basic facts on the representation theory of SU{2) [2\ and the theory of 
Clebsch-Gordan coefficients [H [IT]. For / G = {O, i, 1, §, 2, . . . }, let Vi be the space of all 
homogeneous polynomials of degree 21 in two variables x, y. We can define a representation 
of the group SU (2) on Vi by setting 



[T' {g)P] ix,y)= P{ax + cy, hx + dy) 
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for g = ^) G SU{2) and P G Vi; the set {T' : / € ^2} form a complete list of irreducible 

representations of S{U{2). In Vi we introduce the basis = \J{^^^''^'' V^^^ i |j| ^ ^ (where 
this means that j = —I, —/ + !,...,/) and we suppose that Vi is endowed with a scalar product 
in which this basis is orthonormal. This basis behaves nicely with respect to the restriction 
of Vi to the subgroup U{1) = {{'^ Jl,) : \a\ = l}: if g = (q „-i ) then T\g)il^^j = a-2j>j, that 
is spans an ?7(l)-invariant one dimensional subspace of Vi (corresponding to the character 
o 1-^ a~'^^) and for different values of j we get different characters of U{1). The tensor product 
of two irreducible representations of SU{2) decomposes without multiplicity and according to 
the Clebsch-Gordan formula : 

h+h 

Vi,^Vi,= Vi. (41) 

l=\ll-l2\ 

In the tensor product VJj Vi^ we may introduce two orthonormal bases: the first is simply 
V'j^ 'S^V'jv liil — ^1) lial < h- But we can also introduce a basis that behaves nicely with respect 
to the Clebsch-Gordan decomposition (j4ip : in each subspace Vi in the right hand side of ()4ip we 
can choose a vector corresponding to V'j, and we denote this vector by These vectors are 

uniquely determined by the requirement that they form an orthonormal basis plus the condition 

(<A^'^'\VS ®Vh,)>0. (42) 

Indeed, this additional condition determine a unique unitary operator that intertwines Vi 
and Vi^ ®Vi^. The vectors of the second basis may be expressed in term of the vectors of the 
first basis: 

^''''= E E C^&^^^n- (43) 

jl=-h j2 = -l2 

The numbers C^''?''- are called Clebsch-Gordan coefficients. Note that we have C^''?'i- = 
if ji + J2 / j- 



4.2 The reciprocity between the Gelfand pair {Sn, Sn-m x S^) and the Clebsch- 
Gordan decomposition 

The representation theory of the classical groups GL{n,C),U{n) and SU{n) can be obtained 
from the representation theory of the symmetric group, using the so called Schur-Weyl reci- 
procity; see |25]. In this paper, we use James version of this reciprocity [12], section 26, which is 
quite useful for our purposes. Set ^4 = {1, 2, . . . , /i} and i? = {/i + 1, /i + 2, . . . , n}, as in section 
[Ql For /i = 0, 1, . . . , n we define a hnear map Xh ■ M"""^-™ Vn-m (g) Vm by: if X G J^^ and 

2 2 

\X r\ B\ = w then we set 

1 n — m m 



y \h—m+wJ \w) 



In other words, the exponents of xi,yi, X2,y2 are equal respectively to the cardinalities of 
the sets X'^' n A, r\B,Xr\A,Xr\B. The following decomposition is obvious: 
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/i=0 

Moreover, ImXh is the subspace of Res^^|^j ( Vn-m (g) F™) corresponding to the character 
a 1-^ a^'*~". Clearly, 

a^B)] = ( ^ V"" ~ ^1 x^-'^+'^yr'"-'^ ® x^-'^yf (44) 
\m — wj \ w J 

and the restriction of -^/j to the space of S^-h x S/i-invariant vectors in M^'"^'"^ (that is 
) is injective. 



Lemma 4.1. For every f G M" w;e /ia?;e 

d d d d 

Xhdf = {xi- h yiTr—)Xhf and Xhd* f = (x2^ h y2^)Xhf. 

OX2 oy2 oxi ay I 

Proof. First of all, suppose that / = am-w{-A) ® aw{B). Then we have 

m—w 

(A) (g) ayj{B)] =Xh[{h -m + w + l) {A)®a^{B) 
+ {n-h-w + l)am~w{A) ® a^-iiB)] 

={h-m + w + l)( ^ \('^~^\x\-"'+'"+^y'^-'"-^®x"-^y^ 
\m — w — IJ \ w J 

+ {n-h-w + l)( ^ \('^~^\x^,-'^+'"y'^~'"-^+^ ®x"y^-^ 
\m — wJ \w — IJ 

={m-w)( ^ V"" " ^V?""'^"'^^yr"'~^ ^^"'""^^ 
\m — w J \ w J 

+ w[ \x^ ^ y^ ^0X2 ^2 

\m — w J \ w J 

d d 

= {xiT, h yi-;^)Xh[am-w{A) ® (Tyj{B)]. 

OX2 oy2 

Therefore we have proved the first identity when / is Sn-h x S'h-invariant. Now let be 
the orthogonal projection from TVf"-™.™- onto the subspace of Sn-h x S'/i-invariant functions. 
Clearly, if \X n B\ = w then Pm^x = / h \(n-h\ '^m-w{^) ^ (^w{B). From this fact it follows 

Km — w/ \ w J 

that Pm-id = dPm and XhPm = -^h- Therefore, for any / G M"""^'*" we have 

Xhdf = XhPm-ldf = XhdPmf ={X1T^+ yiT^)XhPmf = {^l^ + yij—)XJ. 

0x2 oy2 0x2 oy2 

The proof for d* is the same. □ 

The proof of the following Lemma is easy. 

Lemma 4.2. The operators {xi-£^+yi-^) ■ V-a-ra (gV^ Vn-m+i (gVm-i and {x2-£^+y2-^) ■ 
Vn-m Vm. — > Vn-m-1 (g Vm+1 commute with the action of SU(2). 
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Now we can write the reciprocal of [3] in Theorem! 
Corollary 4.3. The Clebsch-Gordan decomposition (|41|) may be also written in the form 



V, 



n — m 
2 



min{n-m,m} / q q \ m-k 



(Vn^ 14) n Ker(xi — + yi— ; 
2 2 dx2 oy2 



where the space (Vn^k ^Vk) nKer(xig|^ + ^1^) isomorphic to V^-k (^'>^d coincides with 

®n—k -y Qn—k,k 
h=k '^h'J 

Theorem 4.4. 



n — m m n r, 

Q 2 ' 2 ' 2 



{h - ky.{n -h - k)\{m - ky.{n -m - k)l{n - 2k + 1) 
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(m — w)\{h — m + w)\{n — h — w)lw\{n — k + l)\k\ 
Proof. First of all, from Corollary 12.71 and the definition of X^, we get 



Ek{n — h,h,m,w) (45) 



mm{n-h,m} . ^ \ _ 

Xfi^(n, h,m, k) = } Edn — h,h,m,w)[ )( |x 

^-^ \m — w I \ w 

w=msLx{0 ,— h+m} 



1 



' n—m 
.h—m+w 



)0 



(46) 



Moreover, Corollary 14.31 implies that Xh^(n,h,m,k) G ^f-fc H ImA'/j C V„-m (8) Vm. and 
therefore, since ImX^ corresponds to the SU{1) character a i— > a^^~^ , there exists A G C such 

n~~m m n 

that: Xh^(n, h,m, k) = A(/)n^,'^ '^ . To compute A, first observe that from (HUI) and the 

2 " 

orthogonality relations ([3]) it follows that 



{n - k + l)l{h - k)\{n - h - k)lk\ 



II ^ ^ ' ' ' ^iiV(„_™)/2®v„/2 (^n-2k + l){m-k + l)k{n-m-k + l)k 

{h - k + l)kin - h - k + 1) 
{m — k)\{n — m — k)l 



The positivity condition (I42p is satisfied if and only if the coefficient of ip„3m ^ '4'EL-h 

2 

jXj^^{n,k,m,k) is positive. From the symmetry relation (j37p and from ([3]) we get that this 
coefficient is a positive multiple of 



—E}.{n — k,k,m,m — k) = -^Ek{n — m, m, k, 0) 
and therefore we must have 



k\(n — m — k + 1)^. 
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Xh^{n,h,m,k) =(-1 



{n - k + ly.jh - ky.jn - h - k)\k\ 
y {n-2k + l)(m - k + l)fc(n - m - k + l)k ^^^^ 
^ (/t - + l)fc(n - /t - + l)fc ^ii^,^,^-fc 
{m — k)\{n — m — k)l 

Then (gS]) follows from and (H7D. 

□ 

We may also write (j45p in the following form: 



Em{a,b, c,x) 

x\{c — x)l{b — c + x)\{a — x)\{a + b — m + l)!m! 
(c — m)!(a + 6 — c — m)!(a — m)!(6 — m)!(a + 6 — 2m + 1) 

we have used the identity Ek{n — h, h, m, w) = {—l)^Ek{m, n — m, h,m — w), which follow 
from (|35]) and (f37l) . to transform the right hand side of ([451) . 

4.3 The reciprocity between Af^ and Va 

Now set y = {1, 2, . . . , m} and Z = {m + 1, m + 2, . . . , n}, as in the proof of Lemma l3.10[ Set 
Va = (8) (X) • • • (8 . Following the last paragraph of [12] , we define a linear operator 

2 2 2 ^ 

Xm-M"" ^ Va by setting, if A G 9.x{Y) x J7a-i(^), with 1 = (/i, ^2, • • • , h) 11" m, 1 < a (see ([30])) 
In particular, 



b a a + h 



A'„[cTi(F)®aa-i(^)] = f. ^.V ^ .\x'lyl'-'^(^x'iy'i'-''®...®x'l:yl^-'\ 

Clearly we have 

n 
m=0 

and this is also the decomposition of V^ under the action of SU{1): IniXm corresponds to 
the character a i— > a^™""". 

Now we sketch the construction reciprocal to ()34p . We do not use the general results of James 
(that indeed might be used to study more general settings) and we base our considerations 
simply on the results in section 14.21 and the tree method. We define inductively a subspace 
V{T{a),%{r)) ^ yn_fc of Va as follows. Suppose that we have defined V{T{a.),T/{r)) ^ Vg^_- 
and y(r"(a),r/'(r)) ^ Vn-a, .. Then V{T{a),Z{r)) is the subspace of V{T'{a),X!{r)) ® 

2 

V(T" (a),Tj." (r)) isomorphic to V^-fc- The basis of the induction (i.e. the case of a tree of height 
1) is given by the Clebsch-Gordan decomposition. Clearly, 
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K.= 0n^(a),rr(r)) 
r(k) 

where the sum is over aU representation labelings. Then we define '0(T(a), m, T(k)) as 

——k 

the vector in V {T {a) ,%{y)) corresponding to tpi_^- Since it is defined up to a multiphcative 
constant, we may suppose that it is given by a repeated apphcation of (I43p . That is, 

min{n— a^— j,m— i} 



a+ . n — a 



V'(T(a),m,T(k))= V Cj /Jl} \ V'(^'(a), m-u;, T'(k))0V(r"(a), T' 

^ — ' m+ui, — j=^— ui,-j— m 

Then an easy induction yields: 
Proposition 4.5. 1. For every representation labeling 7^(k) we have: 

n—k 



m=fc 



'n~k,k 



VKe /laue <%'m^('^(a), rn,, T(k)) = A'(/'(T(a), m, T(k)), where the constant A can 5e recur- 
sively computed by: 



A = A'A"x 

{w — j)l{m — i — w)\{af- — m — i + w)\{n — — w — j)\{n — k — i — j + l)\(k — i — j)\ 
(m — k)l(n — m — ky.(n — a^i- — k — j + i)l(af- — k — i + j)\{n — 2k + 1) 

and A' and A" are the constants for T and T" . 

This Proposition shows that the tree method for the Sn-m x ^m-intertwining functions in 
is equivalent to the tree method for the Wigner's coefficients (or 3nj-coefficients) for SU{2). 
For the latter, we refer to [Tl [T3l[29] . 
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